Floating-point Research Articles

Cloudy Weather Prediction Using CNN Models and Satellite Images

Abstract: Cloudy weather classification is a vital task in meteorology and remote sensing, facilitating various applications such as weather forecasting, climate monitoring, and environmental analysis. In this study, we explore the application of convolutional neural network (CNN) techniques for classifying cloudy weather conditions using the Cloudy Weather Dataset sourced from Kaggle. The primary CNN architectures investigated include AlexNet, LeNet, and ResNet. The dataset undergoes preprocessing steps, including resizing and normalization to floating-point representation. Additionally, for calculating cloud cover percentage, the images are processed through grayscaling followed by thresholding. The performance of each CNN model is evaluated based on metrics of accuracy, that is providing insights into their effectiveness for cloudy weather classification

Floating-Point Embedding: Enhancing the Mathematical Comprehension of Large Language Models

The processing and comprehension of numerical information in natural language represent pivotal focal points of scholarly inquiry. Across diverse applications spanning text analysis to information retrieval, the adept management and understanding of the numerical content within natural language are indispensable in achieving task success. Specialized encoding and embedding techniques tailored to numerical data offer an avenue toward improved performance in tasks involving masked prediction and numerical reasoning, inherently characterized by numerical values. Consequently, treating numbers in text merely as words is inadequate; their numerical semantics must be underscored. Recent years have witnessed the emergence of a range of specific encoding methodologies designed explicitly for numerical content, demonstrating promising outcomes. We observe similarities between the Transformer architecture and CPU architecture, with symmetry playing a crucial role. In light of this observation and drawing inspiration from computer system theory, we introduce a floating-point representation and devise a corresponding embedding module. The numerical representations correspond one-to-one with their semantic vector values, rendering both symmetric regarding intermediate transformation methods. Our proposed methodology facilitates the more comprehensive encoding and embedding of numerical information within a predefined precision range, thereby ensuring a distinctive encoding representation for each numerical entity. Rigorous testing on multiple encoder-only models and datasets yielded results that stand out in terms of competitiveness. In comparison to the default embedding methods employed by models, our approach achieved an improvement of approximately 3.8% in Top-1 accuracy and a reduction in perplexity of approximately 0.43. These outcomes affirm the efficacy of our proposed method. Furthermore, the enrichment of numerical semantics through a more comprehensive embedding contributes to the augmentation of the model’s capacity for semantic understanding.

An efficient segmentation model for abnormal chicken droppings recognition based on improved deep dual-resolution network.

The characteristics of chicken droppings are closely linked to their health status. In prior studies, chicken droppings recognition is treated as an object detection task, leading to challenges in labeling and missed detection due to the diverse shapes, overlapping boundaries, and dense distribution of chicken droppings. Additionally, the use of intelligent monitoring equipment equipped with edge devices in farms can significantly reduce manual labor. However, the limited computational power of edge devices presents challenges in deploying real-time segmentation algorithms for field applications. Therefore, this study redefines the task as a segmentation task, with the main objective being the development of a lightweight segmentation model for the automated monitoring of abnormal chicken droppings. A total of 60 Arbor Acres broilers were housed in 5 specific pathogen-free cages for over 3 weeks, and 1650 RGB images of chicken droppings were randomly divided into training and testing sets in an 8:2 ratio to develop and test the model. Firstly, by incorporating the attention mechanism, multi-loss function and auxiliary segmentation head, the segmentation accuracy of the DDRNet was enhanced. Then, by employing the group convolution and an advanced knowledge distillation algorithm, a lightweight segmentation model named DDRNet-s-KD was obtained, which achieved a mean Dice coefficient (mDice) of 79.43% and an inference speed of 86.10 frames per second (FPS), showing a 2.91% and 61.2% increase in mDice and FPS compared to the benchmark model. Furthermore, the DDRNet-s-KD model was quantized from 32-bit floating-point values to 8-bit integers and then converted to TensorRT format. Impressively, the weight size of the quantized model was only 13.7 MB, representing an 82.96% reduction compared to the benchmark model. This makes it well-suited for deployment on the edge device, achieving an inference speed of 137.51 FPS on Jetson Xavier NX. In conclusion, the methods proposed in this study show significant potential in monitoring abnormal chicken droppings and can provide an effective reference for the implementation of other agricultural embedded systems.

A Hardware-Based Orientation Detection System Using Dendritic Computation

Studying how objects are positioned is vital for improving technologies like robots, cameras, and virtual reality. In our earlier papers, we introduced a bio-inspired artificial visual system for orientation detection, demonstrating its superiority over traditional systems with higher recognition rates, greater biological resemblance, and increased resistance to noise. In this paper, we propose a hardware-based orientation detection system (ODS). The ODS is implemented by a multiple dendritic neuron model (DNM), and a neuronal pruning scheme for the DNM is proposed. After performing the neuronal pruning, only the synapses in the direct and inverse connections states are retained. The former can be realized by a comparator, and the latter can be replaced by a combination of a comparator and a logic NOT gate. For the dendritic function, the connection of synapses on dendrites can be realized with logic AND gates. Then, the output of the neuron is equivalent to a logic OR gate. Compared with other machine learning methods, this logic circuit circumvents floating-point arithmetic and therefore requires very little computing resources to perform complex classification. Furthermore, the ODS can be designed based on experience, so no learning process is required. The superiority of ODS is verified by experiments on binary, grayscale, and color image datasets. The ability to process data rapidly owing to advantages such as parallel computation and simple hardware implementation allows the ODS to be desirable in the era of big data. It is worth mentioning that the experimental results are corroborated with anatomical, physiological, and neuroscientific studies, which may provide us with a new insight for understanding the complex functions in the human brain.

Expressive power of ReLU and step networks under floating-point operations

The study of the expressive power of neural networks has investigated the fundamental limits of neural networks. Most existing results assume real-valued inputs and parameters as well as exact operations during the evaluation of neural networks. However, neural networks are typically executed on computers that can only represent a tiny subset of the reals and apply inexact operations, i.e., most existing results do not apply to neural networks used in practice. In this work, we analyze the expressive power of neural networks under a more realistic setup: when we use floating-point numbers and operations as in practice. Our first set of results assumes floating-point operations where the significand of a float is represented by finite bits but its exponent can take any integer value. Under this setup, we show that neural networks using a binary threshold unit or ReLU can memorize any finite input/output pairs and can approximate any continuous function within an arbitrary error. In particular, the number of parameters in our constructions for universal approximation and memorization coincides with that in classical results assuming exact mathematical operations. We also show similar results on memorization and universal approximation when floating-point operations use finite bits for both significand and exponent; these results are applicable to many popular floating-point formats such as those defined in the IEEE 754 standard (e.g., 32-bit single-precision format) and bfloat16.

Floating-Point Approximation Enabling Cost-Effective and High-Precision Digital Implementation of FitzHugh-Nagumo Neural Networks.

The study of neuron interactions and hardware implementations are crucial research directions in neuroscience, particularly in developing large-scale biological neural networks. The FitzHugh-Nagumo (FHN) model is a popular neuron model with highly biological plausibility, but its complexity makes it difficult to apply at scale. This paper presents a cost-saving and improved precision approximation algorithm for the digital implementation of the FHN model. By converting the computational data into floating-point numbers, the original multiplication calculations are replaced by adding the floating-point exponent part and fitting the mantissa part with piecewise linear. In the hardware implementation, shifters and adders are used, greatly reducing resource overhead. Implementing FHN neurons by this approximation calculations on FPGA reduces the normalized root mean square error (RMSE) to 3.5% of the state-of-the-art (SOTA) while maintaining a performance overhead ratio improvement of 1.09 times. Compared to implementations based on approximate multipliers, the proposed method achieves a 20% reduction in error at the cost of a 2.8% increase in overhead.This model gained additional biological properties compared to LIF while reducing the deployment scale by only 9%. Furthermore, the hardware implementation of nine coupled circular networks with eight nodes and directional diffusion was carried out to demonstrate the algorithm's effectiveness on neural networks. The error decreased to 60% compared to the single neuron of the SOTA. This hardware-friendly algorithm allows for the low-cost implementation of high-precision hardware simulation, providing a novel perspective for studying large-scale, biologically plausible neural networks.

Multi-GPU parallel acceleration scheme for meshfree peridynamic simulations

Peridynamics (PD) using integral equations has unique advantages in modeling structural damage evolution, but the expensive computational and memory costs associated with the nonlocal nature of PD severely limit its engineering applications. This study presents a CUDA-based multi-GPU parallel scheme for meshfree PD simulation, which is compatible with dual-horizon PD supporting multiscale spatial discretization. Efficient utilization of computing resources across all available GPUs is achieved by rationalizing data structures across multiple GPUs and using the CUDA memory model. Using atomic operations to avoid computational dual-bond forces further improves code performance. The classical Kalthoff-Winkler experiment and compact-tensile fatigue test were first carried out using 3 NVIDIA Tesla T4 GPUs. Two examples showed that the proposed scheme significantly accelerates the PD model and that the acceleration enhances approximately linearly with the number of GPUs. The speedup of the PD interaction force computation reaches up to 560.7x. Compared to double-precision floating-point values, the GPU performs much better in handling single-precision values. Finally, a PD simulation of rail head heat conduction under moving heat sources with nearly 76 million particles was realized, demonstrating the potential of the proposed scheme to support large-scale PD simulations.

Hybrid Precision Floating-Point (HPFP) Selection to Optimize Hardware-Constrained Accelerator for CNN Training.

The rapid advancement in AI requires efficient accelerators for training on edge devices, which often face challenges related to the high hardware costs of floating-point arithmetic operations. To tackle these problems, efficient floating-point formats inspired by block floating-point (BFP), such as Microsoft Floating Point (MSFP) and FlexBlock (FB), are emerging. However, they have limited dynamic range and precision for the smaller magnitude values within a block due to the shared exponent. This limits the BFP's ability to train deep neural networks (DNNs) with diverse datasets. This paper introduces the hybrid precision (HPFP) selection algorithms, designed to systematically reduce precision and implement hybrid precision strategies, thereby balancing layer-wise arithmetic operations and data path precision to address the shortcomings of traditional floating-point formats. Reducing the data bit width with HPFP allows more read/write operations from memory per cycle, thereby decreasing off-chip data access and the size of on-chip memories. Unlike traditional reduced precision formats that use BFP for calculating partial sums and accumulating those partial sums in 32-bit Floating Point (FP32), HPFP leads to significant hardware savings by performing all multiply and accumulate operations in reduced floating-point format. For evaluation, two training accelerators for the YOLOv2-Tiny model were developed, employing distinct mixed precision strategies, and their performance was benchmarked against an accelerator utilizing a conventional brain floating point of 16 bits (Bfloat16). The HPFP selection, employing 10 bits for the data path of all layers and for the arithmetic of layers requiring low precision, along with 12 bits for layers requiring higher precision, results in a 49.4% reduction in energy consumption and a 37.5% decrease in memory access. This is achieved with only a marginal mean Average Precision (mAP) degradation of 0.8% when compared to an accelerator based on Bfloat16. This comparison demonstrates that the proposed accelerator based on HPFP can be an efficient approach to designing compact and low-power accelerators without sacrificing accuracy.

Accurate Horner methods in real and complex floating-point arithmetic

Accurate Horner methods in real and complex floating-point arithmetic

Point-to-Spike Residual Learning for Energy-Efficient 3D Point Cloud Classification

Spiking neural networks (SNNs) have revolutionized neural learning and are making remarkable strides in image analysis and robot control tasks with ultra-low power consumption advantages. Inspired by this success, we investigate the application of spiking neural networks to 3D point cloud processing. We present a point-to-spike residual learning network for point cloud classification, which operates on points with binary spikes rather than floating-point numbers. Specifically, we first design a spatial-aware kernel point spiking neuron to relate spiking generation to point position in 3D space. On this basis, we then design a 3D spiking residual block for effective feature learning based on spike sequences. By stacking the 3D spiking residual blocks, we build the point-to-spike residual classification network, which achieves low computation cost and low accuracy loss on two benchmark datasets, ModelNet40 and ScanObjectNN. Moreover, the classifier strikes a good balance between classification accuracy and biological characteristics, allowing us to explore the deployment of 3D processing to neuromorphic chips for developing energy-efficient 3D robotic perception systems.

Integer Is Enough: When Vertical Federated Learning Meets Rounding

Vertical Federated Learning (VFL) is a solution increasingly used by companies with the same user group but differing features, enabling them to collaboratively train a machine learning model. VFL ensures that clients exchange intermediate results extracted by their local models, without sharing raw data. However, in practice, VFL encounters several challenges, such as computational and communication overhead, privacy leakage risk, and adversarial attack. Our study reveals that the usage of floating-point (FP) numbers is a common factor causing these issues, as they can be redundant and contain too much information. To address this, we propose a new architecture called rounding layer, which converts intermediate results to integers. Our theoretical analysis and empirical results demonstrate the benefits of the rounding layer in reducing computation and memory overhead, providing privacy protection, preserving model performance, and mitigating adversarial attacks. We hope this paper inspires further research into novel architectures to address practical issues in VFL.

FiniteFieldSolve: Exactly solving large linear systems in high-energy theory

Large linear systems play an important role in high-energy theory, appearing in amplitude bootstraps and during integral reduction. This paper introduces FiniteFieldSolve, a general-purpose toolkit for exactly solving large linear systems over the rationals. The solver interfaces directly with Mathematica, is straightforward to install, and seamlessly replaces Mathematica's native solvers. In testing, FiniteFieldSolve is approximately two orders of magnitude faster than Mathematica and uses an order of magnitude less memory. The package also compares favorably against other public solvers in FiniteFieldSolve's intended use cases. As the name of the package suggests, solutions are obtained via well-known finite field methods. These methods suffer from introducing an inordinate number of modulo (or integer division) operations with respect to different primes. By automatically recompiling itself for each prime, FiniteFieldSolve converts the division operations into much faster combinations of instructions, dramatically improving performance. The technique of compiling the prime can be applied to any finite field solver, where the time savings will be solver dependent. The operation of the package is illustrated through a detailed example of an amplitude bootstrap. Program summaryProgram Title:FiniteFieldSolveCPC Library link to program files:https://doi.org/10.17632/ntxvp58mjg.1Developer's repository link:https://github.com/jfmangan/FiniteFieldSolveLicensing provisions: GPLv3Programming language: Mathematica, C++Nature of problem: Exactly solving large linear systems over the rationals occurs in various settings in high-energy theory, for example when performing integral reduction or bootstrapping an amplitude.Solution method: The linear system is solved by repeatedly row reducing over different finite fields (see Ref. [1] and references therein). Finite fields avoid the intermediary expression swell inherent to arbitrary precision rationals and bypass roundoff errors from floating point numbers. A downside to using modular arithmetic is that it introduces a tremendous number of integer divisions, but this can be mitigated by compiling the divisions down to simpler instructions. The solver is designed to handle arbitrarily dense systems such as those that appear in certain amplitudes bootstraps.

Designing an IEEE-compliant FPU that supports configurable precision for soft processors

Field Programmable Gate Arrays (FPGAs) are commonly used to accelerate floating-point (FP) applications. Although researchers have extensively studied FPGA FP implementations, existing work has largely focused on standalone operators and frequency-optimized designs. These works are not suitable for FPGA soft processors which are more sensitive to latency, impose a lower frequency ceiling, and require IEEE FP standard compliance. We present an open-source floating-point unit (FPU) for FPGA RISC-V soft processors that is fully IEEE compliant with configurable levels of FP precision. Our design emphasizes runtime performance with 25% lower latency in the most common instructions compared to previous works while maintaining efficient resource utilization. Our FPU also allows users to explore various mantissa widths without having to rewrite or recompile their algorithms. We use this to investigate the scalability of our reduced-precision FPU across numerous microbenchmark functions as well as more complex case studies. Our experiments show that applications like the discrete cosine transformation and the Black-Scholes model can realize a speedup of more than 1.35x in conjunction with a 43% and 35% reduction in lookup table and flip-flop resources while experiencing less than a 0.025% average loss in numerical accuracy with a 16-bit mantissa width.

Masking Floating-Point Number Multiplication and Addition of Falcon

In this paper, we provide the first masking scheme for floating-point number multiplication and addition to defend against recent side-channel attacks on Falcon’s pre-image vector computation. Our approach involves a masked nonzero check gadget that securely identifies whether a shared value is zero. This gadget can be utilized for various computations such as rounding the mantissa, computing the sticky bit, checking the equality of two values, and normalizing a number. To support the masked floating-point number addition, we also developed a masked shift and a masked normalization gadget. Our masking design provides both first- and higherorder mask protection, and we demonstrate the theoretical security by proving the (Strong)-Non-Interference properties in the probing model. To evaluate the performance of our approach, we implemented unmasked, first-order, and second-order algorithms on an Arm Cortex-M4 processor, providing cycle counts and the number of random bytes used. We also report the time for one complete signing process with our countermeasure on an Intel-Core CPU. In addition, we assessed the practical security of our approach by conducting the test vector leakage assessment (TVLA) to validate the effectiveness of our protection. Specifically, our TVLA experiment results for second-order masking passed the test in 100,000 measured traces.

Avoiding breakdown in incomplete factorizations in low precision arithmetic

The emergence of low precision floating-point arithmetic in computer hardware has led to a resurgence of interest in the use of mixed precision numerical linear algebra. For linear systems of equations, there has been renewed enthusiasm for mixed precision variants of iterative refinement. We consider the iterative solution of large sparse systems using incomplete factorization preconditioners. The focus is on the robust computation of such preconditioners in half precision arithmetic and employing them to solve symmetric positive definite systems to higher precision accuracy; however, the proposed ideas can be applied more generally. Even for well-conditioned problems, incomplete factorizations can break down when small entries occur on the diagonal during the factorization. When using half precision arithmetic, overflows are an additional possible source of breakdown. We examine how breakdowns can be avoided and we implement our strategies within new half precision Fortran sparse incomplete Cholesky factorization software. Results are reported for a range of problems from practical applications. These demonstrate that, even for highly ill-conditioned problems, half precision preconditioners can potentially replace double precision preconditioners, although unsurprisingly this may be at the cost of additional iterations of a Krylov solver.

Determining the effect of a floating point on the Falcon digital signature algorithm security

The object of research is digital signatures. The Falcon digital signature scheme is one of the finalists in the NIST post-quantum cryptography competition. Its distinctive feature is the use of floating-point arithmetic. However, floating-point arithmetic has so-called rounding noise, which accumulates during computations and in some cases may lead to significant changes in the processed values. The work considers the problem of using rounding noise to build attacks on implementation. The main result of the study is a novel attack on implementation, which enables the secret key recovery. This attack differs from existing attacks in using two separately secure implementations with different computation orders. As a result of the analysis, the conditions under which secret key recovery is possible were revealed. The attack requires 300,000 signatures and two implementations to recover key. The probability of successful attack ranges from 70 % to 76 %. This probability is explained by the structure of the Gaussian sampling algorithm used in the Falcon digital signature. At the same time, a necessary condition for conducting an attack is identical seed during signature generation. This condition makes the attack more theoretical than practical since the correct implementation of the Falcon makes probability of two identical seeds negligible. However, the possible usage of floating-point noise shows potential existence of additional attack vectors for the Falcon that should be covered in security models. The results could be used in the construction of digital signature security models and their implementation in existing information and communication systems

Security Enabled New Term Weight Measure Technique with Data Driven for Next Generation Mobile Computing Networks

In the field of ASIC and FPGA, Machine Learning (ML) techniques play a major role and become predominant for accurate results for different applications like big data analysis and automotive electronics, and driverless vehicles which are required speed and power savings. Due to increasing the demand for higher accuracy, low power, low area consumption, and higher throughput for the complexity of the designs in the latest technology, the proposed system is fulfilling these demands in ASIC and FPGA domains, reconfigurable hardware architecture has been proposed it consists of an ML-based Support Vector Machine (SVM), high-speed AHB protocol and Floating point (FP) operations and also the system has the flexibility to communicate with I2C and I2S protocols. In order to increase throughput with minimal latency, the proposed architecture with AHB protocol and AHB to APB bridge is incorporated between the fabric dynamically reconfigurable multi-processor (FDPM) and peripherals along with security algorithms using SHA-256bits and AES. In order to perform ML-based applications, the proposed system is incorporated double-precision floating point (DPFP) arithmetic operations. The overall proposed architecture is developed in Verilog HDL and quality checking using the LINT tool and Clock Domain Crossing (CDC) using Spyglass tool and synthesized using DC compiler for ASIC and Vivado Design Suite 2018.1 for FPGA implementation and verification. The entire design is interfaced with the Zynq processor and SDK tool to verify data transfer between hardware and software. The obtained results show the generated custom accelerator is able to compute any complex ML classifiers for a larger amount of data. The obtained results are compared with existing state-of-art results and found that 18 % improvement in throughput, a 21 % improvement in power consumption savings, and a 34 % reduction in latency.

Spatial sensitivity to absorption changes for various near-infrared spectroscopy methods: A compendium review

This compendium review focuses on the spatial distribution of sensitivity to localized absorption changes in optically diffuse media, particularly for measurements relevant to near-infrared spectroscopy. The three temporal domains, continuous wave, frequency domain, and time domain, each obtain different optical data types whose changes may be related to effective homogeneous changes in the absorption coefficient. Sensitivity is the relationship between a localized perturbation and the recovered effective homogeneous absorption change. Therefore, spatial sensitivity maps representing the perturbation location can be generated for the numerous optical data types in the three temporal domains. The review first presents a history of the past 30 years of work investigating this sensitivity in optically diffuse media. These works are experimental and theoretical, presenting one-, two-, and three-dimensional sensitivity maps for different Near-Infrared Spectroscopy methods, domains, and data types. Following this history, we present a compendium of sensitivity maps organized by temporal domain and then data type. This compendium provides a valuable tool to compare the spatial sensitivity of various measurement methods and parameters in one document. Methods for one to generate these maps are provided in Appendix  A , including the code. This historical review and comprehensive sensitivity map compendium provides a single source researchers may use to visualize, investigate, compare, and generate sensitivity to localized absorption change maps.

Numerical Algorithms for Approximation of Fractional Integrals and Derivatives Based on Quintic Spline Interpolation

Numerical algorithms for calculating the left- and right-sided Riemann–Liouville fractional integrals and the left- and right-sided fractional derivatives in the Caputo sense using spline interpolation techniques are derived. The spline of the fifth degree (the so-called quintic spline) is mainly taken into account, but the linear and cubic splines are also considered to compare the quality of the developed method and numerical calculations. The estimation of errors for the derived approximation algorithms is presented. Examples of the numerical evaluation of the fractional integrals and derivatives are executed using 128-bit floating-point numbers and arithmetic routines. For each derived algorithm, the experimental orders of convergence are calculated. Also, an illustrative computational example showing the action of the considered fractional operators on the symmetric function in the interval is presented.

Cryptographic Solutions for Data Security in Cloud Computing:

Due to the recent explosion in the amount of data being created by various social media platforms, e-commerce websites, and other businesses, a paradigm shift from on-site data centers to the cloud is required. Concerns about privacy and secrecy have been a major obstacle to the mainstream adoption of cloud computing. The best approach to protect the confidentiality and privacy of cloud data is by using cryptographic techniques. Researchers have developed several cryptographic algorithms, but they all have lengthy, linear, predictable, memory-intensive execution times. The performance of the CPU, memory, run-time trend, and throughput of the three cryptographic schemes: Enhanced RSA (ERSA), Non-Deterministic Cryptographic Scheme (NCS), and Enhanced Homomorphic Scheme (EHS) are compared using RAsys. The experiment’s results demonstrated that NCS and EHS produced non-linear and non-deterministic run times. Again, NCS and EHS produced the lowest throughput and memory consumption for text and numeric data types when data sizes of 5n*102(KB(∈1,2,4,10,20,40) were processed. However, ERSA produced a run-time trend that was deterministic, linear, and predictable